Huis Verkennen Help
Registreer Inloggen
J1X
/
huginn
Volgen 2
Ster 0
Vork 0

Bladeren bron
First version of the manual deployment guide

Dominik Sander 9 jaren geleden
bovenliggende
b88e20d0a8
commit
0e4c9be723
12 gewijzigde bestanden met toevoegingen van 691 en 7 verwijderingen
Toon Diff Stats
  1. 1 0
      .gitignore
  2. 1 1
      Gemfile
  3. 3 3
      Gemfile.lock
  4. 17 3
      Procfile
  5. 35 0
      config/unicorn.rb.example
  6. 11 0
      doc/README.md
  7. 5 0
      doc/install/README.md
  8. 341 0
      doc/install/installation.md
  9. 68 0
      doc/install/requirements.md
  10. 20 0
      lib/support/logrotate/huginn
  11. 70 0
      lib/support/nginx/huginn
  12. 119 0
      lib/support/nginx/huginn-ssl

+ 1 - 0
.gitignore
Bestand weergeven 

@@ -25,3 +25,4 @@ deployment/cookbooks
25 25 
 .ruby-gemset
26 26 
 .ruby-version
27 27 
 manifest.yml
28 
+config/unicorn.rb

+ 1 - 1
Gemfile
Bestand weergeven 

@@ -120,6 +120,7 @@ end
120 120
121 121 
 group :production do
122 122 
   gem 'rack', '> 1.5.0'
123 
+  gem 'unicorn', '~> 4.9.0'
123 124 
 end
124 125
125 126 
 # Platform requirements.
 
@@ -145,6 +146,5 @@ end
145 146
146 147 
 on_heroku do
147 148 
   gem 'pg'
148 
-  gem 'unicorn'
149 149 
   gem 'rails_12factor', group: :production
150 150 
 end

+ 3 - 3
Gemfile.lock
Bestand weergeven 

@@ -230,7 +230,7 @@ GEM
230 230 
     kaminari (0.16.1)
231 231 
       actionpack (>= 3.0.0)
232 232 
       activesupport (>= 3.0.0)
233 
-    kgio (2.9.2)
233 
+    kgio (2.9.3)
234 234 
     kramdown (1.3.3)
235 235 
     launchy (2.4.2)
236 236 
       addressable (~> 2.3)
 
@@ -472,7 +472,7 @@ GEM
472 472 
     unf (0.1.4)
473 473 
       unf_ext
474 474 
     unf_ext (0.0.7.1)
475 
-    unicorn (4.8.3)
475 
+    unicorn (4.9.0)
476 476 
       kgio (~> 2.6)
477 477 
       rack
478 478 
       raindrops (~> 0.7)
 
@@ -576,7 +576,7 @@ DEPENDENCIES
576 576 
   tzinfo (>= 1.2.0)
577 577 
   tzinfo-data
578 578 
   uglifier (>= 1.3.0)
579 
-  unicorn
579 
+  unicorn (~> 4.9.0)
580 580 
   vcr
581 581 
   webmock (~> 1.17.4)
582 582 
   weibo_2!

+ 17 - 3
Procfile
Bestand weergeven 

@@ -1,13 +1,27 @@
1 
+#################
2 
+# DEVELOPMENT   #
3 
+#################
4 
+
1 5 
 # Procfile for development using the new threaded worker (scheduler, twitter stream and delayed job)
2 6 
 web: bundle exec rails server -b0.0.0.0
3 7 
 jobs: bundle exec rails runner bin/threaded.rb
4 8
5 
-# Possible Profile configuration for production:
6 
-# web: bundle exec unicorn -c config/unicorn/production.rb
9 
+# Old version with separate processes (use this if you have issues with the threaded version)
10 
+# web: bundle exec rails server
11 
+# schedule: bundle exec rails runner bin/schedule.rb
12 
+# twitter: bundle exec rails runner bin/twitter_stream.rb
13 
+# dj: bundle exec script/delayed_job run
14 
+
15 
+#################
16 
+# PRODUCTION    #
17 
+#################
18 
+
19 
+# Using the threaded worker (consumes less RAM but can run slower)
20 
+# web: bundle exec unicorn -c config/unicorn.rb
7 21 
 # jobs: bundle exec rails runner bin/threaded.rb
8 22
9 23 
 # Old version with separate processes (use this if you have issues with the threaded version)
10 
-# web: bundle exec rails server -b0.0.0.0
24 
+# web: bundle exec unicorn -c config/unicorn.rb
11 25 
 # schedule: bundle exec rails runner bin/schedule.rb
12 26 
 # twitter: bundle exec rails runner bin/twitter_stream.rb
13 27 
 # dj: bundle exec script/delayed_job run

+ 35 - 0
config/unicorn.rb.example
Bestand weergeven 

@@ -0,0 +1,35 @@
1 
+wd = "/home/huginn/huginn"
2 
+
3 
+app_path = wd
4 
+
5 
+worker_processes 2
6 
+preload_app true
7 
+timeout 180
8 
+listen "#{wd}/tmp/sockets/unicorn.socket"
9 
+
10 
+working_directory app_path
11 
+
12 
+rails_env = ENV['RAILS_ENV'] || 'production'
13 
+
14 
+# Log everything to one file
15 
+stderr_path "log/unicorn.log"
16 
+stdout_path "log/unicorn.log"
17 
+
18 
+# Set master PID location
19 
+pid "#{wd}/tmp/pids/unicorn.pid"
20 
+
21 
+before_fork do |server, worker|
22 
+  ActiveRecord::Base.connection.disconnect!
23 
+  old_pid = "#{server.config[:pid]}.oldbin"
24 
+  if File.exist?(old_pid) && server.pid != old_pid
25 
+    begin
26 
+      Process.kill("QUIT", File.read(old_pid).to_i)
27 
+    rescue Errno::ENOENT, Errno::ESRCH
28 
+      # someone else did our job for us
29 
+    end
30 
+  end
31 
+end
32 
+
33 
+after_fork do |server, worker|
34 
+  ActiveRecord::Base.establish_connection
35 
+end

+ 11 - 0
doc/README.md
Bestand weergeven 

@@ -0,0 +1,11 @@
1 
+# Documentation
2 
+
3 
+## User documentation
4 
+
5 
+- [Check out Huginn with Docker](./tobewritten.md) Run a local Huginn installation using Docker
6 
+
7 
+
8 
+## Administrator documentation
9 
+
10 
+- [Install](install/README.md) Requirements, directory structures and installation from source.
11 
+- [Update](update/README.md) Update guides to upgrade your installation.

+ 5 - 0
doc/install/README.md
Bestand weergeven 

@@ -0,0 +1,5 @@
1 
+# Installation
2 
+
3 
+- [Requirements](requirements.md) Software and hardware requirements to run the Huginn installation
4 
+- [Install](installation.md) Installation guide for Ubundu/Debian
5 
+- [Update](update.md) Update an existing Huginn installation

+ 341 - 0
doc/install/installation.md
Bestand weergeven 

@@ -0,0 +1,341 @@
1 
+# Installation from source
2 
+
3 
+
4 
+## Important Notes
5 
+
6 
+This guide is long because it covers many cases and includes all commands you need.
7 
+
8 
+This installation guide was created for and tested on **Debian/Ubuntu** operating systems. Please read [doc/install/requirements.md](./requirements.md) for hardware and operating system requirements.
9 
+
10 
+This is the official installation guide to set up a production server. To set up a **development installation** or for many other installation options please see [the getting started section of the readme](https://github.com/cantino/huginn#getting-started).
11 
+
12 
+The following steps have been known to work. Please **use caution when you deviate** from this guide. Make sure you don't violate any assumptions Huginn makes about its environment. For example many people run into permission problems because they change the location of directories or run services as the wrong user.
13 
+
14 
+If you find a bug/error in this guide please **submit a merge request**.
15 
+
16 
+## Overview
17 
+
18 
+The Huginn installation consists of setting up the following components:
19 
+
20 
+1. Packages / Dependencies
21 
+1. Ruby
22 
+1. System Users
23 
+1. Database
24 
+1. Huginn
25 
+1. Nginx
26 
+
27 
+## 1. Packages / Dependencies
28 
+
29 
+`sudo` is not installed on Debian by default. Make sure your system is
30 
+up-to-date and install it.
31 
+
32 
+    # run as root!
33 
+    apt-get update -y
34 
+    apt-get upgrade -y
35 
+    apt-get install sudo -y
36 
+
37 
+**Note:** During this installation some files will need to be edited manually. If you are familiar with vim set it as default editor with the commands below. If you are not familiar with vim please skip this and keep using the default editor.
38 
+
39 
+    # Install vim and set as default editor
40 
+    sudo apt-get install -y vim
41 
+    sudo update-alternatives --set editor /usr/bin/vim.basic
42 
+
43 
+Import node.js repository (can be skipped on Ubuntu and Debian Jessie):
44 
+
45 
+    curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -
46 
+
47 
+Install the required packages (needed to compile Ruby and native extensions to Ruby gems):
48 
+
49 
+    sudo apt-get install -y build-essential git zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake nodejs graphviz
50 
+
51 
+
52 
+## 2. Ruby
53 
+
54 
+
55 
+The use of Ruby version managers such as [RVM](http://rvm.io/), [rbenv](https://github.com/sstephenson/rbenv) or [chruby](https://github.com/postmodern/chruby) with Huginn in production frequently leads to hard-to-diagnose problems. Version managers are not supported and we strongly advise everyone to follow the instructions below to use a system Ruby.
56 
+
57 
+Remove the old Ruby versions if present:
58 
+
59 
+    sudo apt-get remove -y ruby1.8 ruby1.9
60 
+
61 
+Download Ruby and compile it:
62 
+
63 
+    mkdir /tmp/ruby && cd /tmp/ruby
64 
+    curl -L --progress http://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.2.tar.bz2 | tar xj
65 
+    cd ruby-2.2.2
66 
+    ./configure --disable-install-rdoc
67 
+    make -j`nproc`
68 
+    sudo make install
69 
+
70 
+Install the bundler and foreman gems:
71 
+
72 
+    sudo gem install bundler foreman --no-ri --no-rdoc
73 
+
74 
+## 3. System Users
75 
+
76 
+Create a user for Huginn:
77 
+
78 
+    sudo adduser --disabled-login --gecos 'Huginn' huginn
79 
+
80 
+## 4. Database
81 
+
82 
+Install the database packages
83 
+
84 
+    sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev
85 
+
86 
+    # Pick a MySQL root password (can be anything), type it and press enter,
87 
+    # retype the MySQL root password and press enter
88 
+
89 
+Check the installed MySQL version (remeber if its >= 5.5.3 for the `.env` configuration done later):
90 
+
91 
+    mysql --version
92 
+
93 
+Secure your installation
94 
+
95 
+    sudo mysql_secure_installation
96 
+
97 
+Login to MySQL
98 
+
99 
+    mysql -u root -p
100 
+
101 
+    # Type the MySQL root password
102 
+
103 
+Create a user for Huginn do not type the `mysql>`, this is part of the prompt. Change `$password` in the command below to a real password you pick
104 
+
105 
+    mysql> CREATE USER 'huginn'@'localhost' IDENTIFIED BY '$password';
106 
+
107 
+Ensure you can use the InnoDB engine which is necessary to support long indexes
108 
+
109 
+    mysql> SET storage_engine=INNODB;
110 
+
111 
+    # If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`)
112 
+    # for the setting "innodb = off"
113 
+
114 
+Grant the Huginn user necessary permissions on the database
115 
+
116 
+    mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES ON `huginn_production`.* TO 'huginn'@'localhost';
117 
+
118 
+Quit the database session
119 
+
120 
+    mysql> \q
121 
+
122 
+Try connecting to the new database with the new user
123 
+
124 
+    sudo -u huginn -H mysql -u huginn -p -D huginn_production
125 
+
126 
+    # Type the password you replaced $password with earlier
127 
+
128 
+You should now see `ERROR 1049 (42000): Unknown database 'huginn_production'` which is fine because we will create the database later.
129 
+
130 
+You are done installing the database and can go back to the rest of the installation.
131 
+
132 
+
133 
+## 6. Huginn
134 
+
135 
+### Clone the Source
136 
+
137 
+    # We'll install Huginn into home directory of the user "huginn"
138 
+    cd /home/huginn
139 
+
140 
+    # Clone Huginn repository
141 
+    #sudo -u huginn -H git clone https://github.com/cantino/huginn.git -b master huginn
142 
+    # **FIXME**
143 
+    sudo -u huginn -H git clone https://github.com/dsander/huginn.git -b deployment-guide huginn
144 
+
145 
+    # Go to Huginn installation folder
146 
+    cd /home/huginn/huginn
147 
+
148 
+    # Copy the example Huginn config
149 
+    sudo -u huginn -H cp .env.example .env
150 
+
151 
+    # Create the log/, tmp/pids/ and tmp/sockets/ directories
152 
+    sudo -u huginn mkdir -p log tmp/pids tmp/sockets
153 
+
154 
+    # Make sure Huginn can write to the log/ and tmp/ directories
155 
+    sudo chown -R huginn log/ tmp/
156 
+    sudo chmod -R u+rwX,go-w log/ tmp/
157 
+
158 
+    # Make sure permissions are set correctly
159 
+    sudo chmod -R u+rwX,go-w log/
160 
+    sudo chmod -R u+rwX tmp/
161 
+    sudo -u huginn -H chmod o-rwx .env
162 
+
163 
+    # Copy the example Unicorn config
164 
+    sudo -u huginn -H cp config/unicorn.rb.example config/unicorn.rb
165 
+
166 
+### Install Gems
167 
+
168 
+**Note:** As of bundler 1.5.2, you can invoke `bundle install -jN` (where `N` the number of your processor cores) and enjoy parallel gem installation with measurable difference in completion time (~60% faster). Check the number of your cores with `nproc`. For more information check this [post](http://robots.thoughtbot.com/parallel-gem-installing-using-bundler). First make sure you have bundler >= 1.5.2 (run `bundle -v`) as it addresses some [issues](https://devcenter.heroku.com/changelog-items/411) that were [fixed](https://github.com/bundler/bundler/pull/2817) in 1.5.2.
169 
+
170 
+    sudo -u huginn -H bundle install --deployment --without development test
171 
+
172 
+
173 
+### Configure it
174 
+
175 
+    # Update Huginn config file and follow the instructions
176 
+    sudo -u huginn -H editor .env
177 
+
178 
+If you are using a local MySQL server the database configuration should look like this (use the password of the huginn MySQL user you created earlier):
179 
+
180 
+    DATABASE_ADAPTER=mysql2
181 
+    DATABASE_ENCODING=utf8
182 
+    DATABASE_RECONNECT=true
183 
+    DATABASE_NAME=huginn_production
184 
+    DATABASE_POOL=20
185 
+    DATABASE_USERNAME=huginn
186 
+    DATABASE_PASSWORD=""
187 
+    #DATABASE_HOST=your-domain-here.com
188 
+    #DATABASE_PORT=3306
189 
+    #DATABASE_SOCKET=/tmp/mysql.sock
190 
+
191 
+**Important**: Uncomment the RAILS_ENV setting to run Huginn in the production rails environment
192 
+
193 
+    RAILS_ENV=production
194 
+
195 
+Change the Unicorn config if needed, the [requirements.md](./requirements.md#unicorn-workers) has a section explaining the suggested amount of unicorn workers:
196 
+
197 
+    # Increase the amount of workers if you expect to have a high load instance.
198 
+    # 2 are enough for most use cases, if the server has less then 2GB of RAM
199 
+    # decrease the worker amount to 1
200 
+    sudo -u huginn -H editor config/unicorn.rb
201 
+
202 
+
203 
+**Important Note:** Make sure to edit both `.env` and `unicorn.rb` to match your setup.
204 
+
205 
+**Note:** If you want to use HTTPS, see [Using HTTPS](#using-https) for the additional steps.
206 
+
207 
+
208 
+### Initialize Database
209 
+
210 
+    # Create the database
211 
+    sudo -u huginn -H bundle exec rake db:create RAILS_ENV=production
212 
+
213 
+    # Migrate to the latest version
214 
+    sudo -u huginn -H bundle exec rake db:migrate RAILS_ENV=production
215 
+
216 
+    # Create admin user and example agents
217 
+    sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=production
218 
+
219 
+When done you see `See the Huginn Wiki for more Agent examples!  https://github.com/cantino/huginn/wiki`
220 
+
221 
+**Note:** This will create an initial user, you can set the username and password by supplying it in environmental variables `SEED_USERNAME` and`SEED_PASSWORD` as seen below. If you don't set the password (and it is set to the default one) please wait with exposing Huginn to the public internet until the installation is done and you've logged into the server and changed your password.
222 
+
223 
+    sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=production SEED_USERNAME=admin SEED_PASSWORD=yourpassword
224 
+
225 
+### Install Init Script
226 
+
227 
+Huginn uses foreman to generate the init scripts based on a `Procfile`
228 
+
229 
+Edit the `Procfile` and choose one of the suggested versions for production
230 
+
231 
+    sudo -u huginn -H editor Procfile
232 
+
233 
+**Debian only** Install upstart and reboot the system (skip this step on Ubuntu):
234 
+
235 
+    sudo apt-get install -y --force-yes upstart
236 
+    sudo reboot
237 
+    # After you you logged back in go to Huginn installation folder
238 
+    cd /home/huginn/huginn
239 
+
240 
+Export the init scripts using foreman:
241 
+
242 
+    sudo foreman export upstart -a huginn /etc/init
243 
+
244 
+**Note:** You have to re-export the init script every time you change the configuration in `.env`!
245 
+
246 
+### Setup Logrotate
247 
+
248 
+    sudo cp lib/support/logrotate/huginn /etc/logrotate.d/huginn
249 
+
250 
+### Compile Assets
251 
+
252 
+    sudo -u huginn -H bundle exec rake assets:precompile RAILS_ENV=production
253 
+
254 
+### Start Your Huginn Instance
255 
+
256 
+    sudo start huginn
257 
+
258 
+## 7. Nginx
259 
+
260 
+**Note:** Nginx is the officially supported web server for Huginn. If you cannot or do not want to use Nginx as your web server, the wiki has a page on how to configure [apache](https://github.com/cantino/huginn/wiki/Apache-Huginn-configuration).
261 
+
262 
+### Installation
263 
+
264 
+    sudo apt-get install -y nginx
265 
+
266 
+### Site Configuration
267 
+
268 
+Copy the example site config:
269 
+
270 
+    sudo cp lib/support/nginx/huginn /etc/nginx/sites-available/huginn
271 
+    sudo ln -s /etc/nginx/sites-available/huginn /etc/nginx/sites-enabled/huginn
272 
+
273 
+Make sure to edit the config file to match your setup, if you are running multiple nginx sites remove the `default_server` argument from the `listen` directives:
274 
+
275 
+    # Change YOUR_SERVER_FQDN to the fully-qualified
276 
+    # domain name of your host serving Huginn.
277 
+    sudo editor /etc/nginx/sites-available/huginn
278 
+
279 
+Remove the default nginx site, **if huginn is the only enabled nginx site**:
280 
+
281 
+    sudo rm /etc/nginx/sites-enabled/default
282 
+
283 
+**Note:** If you want to use HTTPS, replace the `huginn` Nginx config with `huginn-ssl`. See [Using HTTPS](#using-https) for HTTPS configuration details.
284 
+
285 
+### Test Configuration
286 
+
287 
+Validate your `huginn` or `huginn-ssl` Nginx config file with the following command:
288 
+
289 
+    sudo nginx -t
290 
+
291 
+You should receive `syntax is okay` and `test is successful` messages. If you receive errors check your `huginn` or `huginn-ssl` Nginx config file for typos, etc. as indicated in the error message given.
292 
+
293 
+### Restart
294 
+
295 
+    sudo service nginx restart
296 
+
297 
+# Done!
298 
+
299 
+### Initial Login
300 
+
301 
+Visit YOUR_SERVER in your web browser for your first Huginn login. The setup has created a default admin account for you. You can use it to log in:
302 
+
303 
+    admin
304 
+    password
305 
+
306 
+
307 
+**Enjoy!**
308 
+
309 
+You can use `sudo start huginn` and `sudo stop huginn` to start and stop Huginn.
310 
+
311 
+## Advanced Setup Tips
312 
+
313 
+### Using HTTPS
314 
+
315 
+To use Huginn with HTTPS:
316 
+
317 
+1. In `.env`:
318 
+    1. Set the `FORCE_SSL` option to `true`.
319 
+1. Use the `huginn-ssl` Nginx example config instead of the `huginn` config:
320 
+    1. `sudo cp lib/support/nginx/huginn-ssl /etc/nginx/sites-available/huginn`
321 
+    1. Update `YOUR_SERVER_FQDN`.
322 
+    1. Update `ssl_certificate` and `ssl_certificate_key`.
323 
+    1. Review the configuration file and consider applying other security and performance enhancing features.
324 
+
325 
+Restart Nginx, export the init script and restart Huginn:
326 
+
327 
+```
328 
+cd /home/huginn/huginn
329 
+sudo service nginx restart
330 
+sudo foreman export upstart -a huginn /etc/init
331 
+sudo restart huginn
332 
+```
333 
+
334 
+Using a self-signed certificate is discouraged, but if you must use it follow the normal directions. Then generate the certificate:
335 
+
336 
+```
337 
+sudo mkdir -p /etc/nginx/ssl/
338 
+cd /etc/nginx/ssl/
339 
+sudo openssl req -newkey rsa:2048 -x509 -nodes -days 3560 -out huginn.crt -keyout huginn.key
340 
+sudo chmod o-r huginn.key
341 
+```

+ 68 - 0
doc/install/requirements.md
Bestand weergeven 

@@ -0,0 +1,68 @@
1 
+# Requirements
2 
+
3 
+## Operating Systems
4 
+
5 
+### Supported Unix distributions
6 
+
7 
+- Ubuntu (12.04 and 14.04)
8 
+- Debian (Jessie and Wheezy)
9 
+
10 
+### Unsupported Unix distributions
11 
+
12 
+- CentOS
13 
+- Red Hat Enterprise Linux
14 
+- OS X
15 
+- Arch Linux
16 
+- Fedora
17 
+- Gentoo
18 
+- FreeBSD
19 
+
20 
+On the above unsupported distributions is still possible to install Huginn. Follow the [installation guide](./installation.md) and substitute the `apt` commands with the corresponding package manager commands of your distribution.
21 
+
22 
+### Non-Unix operating systems such as Windows
23 
+
24 
+Huginn is developed for Unix operating systems.
25 
+Huginn does **not** run on Windows and we have no plans of supporting it in the near future.
26 
+Please consider using a virtual machine to run Huginn.
27 
+
28 
+## Ruby versions
29 
+
30 
+Huginn requires Ruby (MRI) 2.0, 2.1 or 2.2
31 
+You will have to use the standard MRI implementation of Ruby.
32 
+We love [JRuby](http://jruby.org/) and [Rubinius](http://rubini.us/) but Huginn needs several Gems that have native extensions.
33 
+
34 
+## Hardware requirements
35 
+
36 
+### CPU
37 
+
38 
+- _single core_ setups will work but depending on the amount of Huginn Agents and users it will run a bit slower since the application server and background jobs can not run simultaneously
39 
+- _dual core_ setups are the **recommended** system/vps and will work well for a decent amount of Agents
40 
+- 3+ cores can be needed when running multiple DelayedJob workers
41 
+
42 
+### Memory
43 
+
44 
+You need at least 0.5GB of physical and 0.5GB of addressable memory (swap) to install and use Huginn with the default configuration!
45 
+With less memory you need to manually adjust the `Gemfile` and Huginn can respond with internal server errors when accessing the web interface.
46 
+
47 
+- 256MB RAM + 0.5GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the Wiki page about running Huginn on [systems with low memory](https://github.com/cantino/huginn/wiki/Running-Huginn-on-minimal-systems-with-low-RAM-&-CPU-e.g.-Raspberry-Pi)
48 
+- 0.5GB RAM + 0.5GB swap will work relatively well with SSD drives, but can feel a bit slow due to swapping
49 
+- 1GB RAM + 1GB swap will work with two unicorn workers and the threaded background worker
50 
+- **2GB RAM** is the **recommended** memory size, it will support 2 unicorn workers and both the threaded and the old separate workers
51 
+- for each 300MB of additional RAM you can run one extra DelayedJob worker
52 
+
53 
+## Unicorn Workers
54 
+
55 
+It's possible to increase the amount of unicorn workers and this will usually help for to reduce the response time of the applications and increase the ability to handle parallel requests.
56 
+
57 
+For most instances we recommend using: CPU cores = unicorn workers.
58 
+
59 
+If you have a 512MB machine we recommend to configure only one Unicorn worker and use the threaded background worker to prevent excessive swapping.
60 
+
61 
+
62 
+## DelayedJob Workers
63 
+
64 
+A DelayedJob worker is a separate process which runs your Huginn Agents. It fetches Websites, polls external services for updates, etc. Depending on the amount of Agents and the check frequency of those you might need to run more than one worker (like it is done in the threaded setup).
65 
+
66 
+Estimating the amount of workers needed is easy. One worker can perform just one check at a time.
67 
+If you have 60 Agents checking websites every minute which take about 1 second to respond, one worker is fine.
68 
+If you need more Agents or are dealing with slow/unreliable websites/services, you should consider running additional workers.

+ 20 - 0
lib/support/logrotate/huginn
Bestand weergeven 

@@ -0,0 +1,20 @@
1 
+/home/huginn/huginn/log/*.log {
2 
+  daily
3 
+  missingok
4 
+  rotate 180
5 
+  # must use with delaycompress below
6 
+  compress
7 
+  dateext
8 
+
9 
+  # this is important if using "compress" since we need to call
10 
+  # the "lastaction" script below before compressing:
11 
+  delaycompress
12 
+
13 
+  # note the lack of the evil "copytruncate" option in this
14 
+  # config.  Unicorn supports the USR1 signal and we send it
15 
+  # as our "lastaction" action:
16 
+  lastaction
17 
+    pid=/home/huginn/huginn/tmp/pids/unicorn.pid
18 
+    test -s $pid && kill -USR1 "$(cat $pid)"
19 
+  endscript
20 
+}

+ 70 - 0
lib/support/nginx/huginn
Bestand weergeven 

@@ -0,0 +1,70 @@
1 
+## Huginn
2 
+##
3 
+## Lines starting with two hashes (##) are comments with information.
4 
+## Lines starting with one hash (#) are configuration parameters that can be uncommented.
5 
+##
6 
+###################################
7 
+##         configuration         ##
8 
+###################################
9 
+##
10 
+## See installation.md#using-https for additional HTTPS configuration details.
11 
+
12 
+upstream huginn {
13 
+  server unix:/home/huginn/huginn/tmp/sockets/unicorn.socket fail_timeout=0;
14 
+}
15 
+
16 
+## Normal HTTP host
17 
+server {
18 
+  listen 0.0.0.0:80 default_server;
19 
+  listen [::]:80 ipv6only=on default_server;
20 
+  server_name YOUR_SERVER_FQDN; ## Replace this with something like huginn.example.com
21 
+  server_tokens off; ## Don't show the nginx version number, a security best practice
22 
+  root /home/huginn/huginn/public;
23 
+
24 
+  ## Increase this if you want to upload large attachments
25 
+  client_max_body_size 20m;
26 
+
27 
+  ## Individual nginx logs for this Huginn vhost
28 
+  access_log  /var/log/nginx/huginn_access.log;
29 
+  error_log   /var/log/nginx/huginn_error.log;
30 
+
31 
+  location / {
32 
+    ## Serve static files from defined root folder.
33 
+    ## @huginn is a named location for the upstream fallback, see below.
34 
+    try_files $uri $uri/index.html $uri.html @huginn;
35 
+  }
36 
+
37 
+  ## If a file, which is not found in the root folder is requested,
38 
+  ## then the proxy passes the request to the upsteam (huginn unicorn).
39 
+  location @huginn {
40 
+    ## If you use HTTPS make sure you disable gzip compression
41 
+    ## to be safe against BREACH attack.
42 
+    # gzip off;
43 
+
44 
+    proxy_read_timeout      300;
45 
+    proxy_connect_timeout   300;
46 
+    proxy_redirect          off;
47 
+
48 
+    proxy_set_header    Host                $http_host;
49 
+    proxy_set_header    X-Real-IP           $remote_addr;
50 
+    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
51 
+    proxy_set_header    X-Forwarded-Proto   $scheme;
52 
+    proxy_set_header    X-Frame-Options     SAMEORIGIN;
53 
+
54 
+    proxy_pass http://huginn;
55 
+  }
56 
+
57 
+  ## Enable gzip compression as per rails guide:
58 
+  ## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
59 
+  ## WARNING: If you are using relative urls remove the block below
60 
+  ## See config/application.rb under "Relative url support" for the list of
61 
+  ## other files that need to be changed for relative url support
62 
+  location ~ ^/(assets)/ {
63 
+    root /home/huginn/huginn/public;
64 
+    gzip_static on; # to serve pre-gzipped version
65 
+    expires max;
66 
+    add_header Cache-Control public;
67 
+  }
68 
+
69 
+  error_page 502 /502.html;
70 
+}

+ 119 - 0
lib/support/nginx/huginn-ssl
Bestand weergeven 

@@ -0,0 +1,119 @@
1 
+## Huginn
2 
+##
3 
+## Modified from nginx http version
4 
+## Modified from http://blog.phusion.nl/2012/04/21/tutorial-setting-up-gitlab-on-debian-6/
5 
+## Modified from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
6 
+## Modified from https://github.com/gitlabhq/gitlabhq/blob/master/lib/support/nginx/gitlab-ssl
7 
+##
8 
+## Lines starting with two hashes (##) are comments with information.
9 
+## Lines starting with one hash (#) are configuration parameters that can be uncommented.
10 
+##
11 
+###################################
12 
+##         configuration         ##
13 
+###################################
14 
+##
15 
+## See installation.md#using-https for additional HTTPS configuration details.
16 
+
17 
+upstream huginn {
18 
+  server unix:/home/huginn/huginn/tmp/sockets/unicorn.socket fail_timeout=0;
19 
+}
20 
+
21 
+## Redirects all HTTP traffic to the HTTPS host
22 
+server {
23 
+  listen 0.0.0.0:80;
24 
+  listen [::]:80 ipv6only=on default_server;
25 
+  server_name YOUR_SERVER_FQDN; ## Replace this with something like huginn.example.com
26 
+  server_tokens off; ## Don't show the nginx version number, a security best practice
27 
+  return 301 https://$server_name$request_uri;
28 
+  access_log  /var/log/nginx/huginn_access.log;
29 
+  error_log   /var/log/nginx/huginn_error.log;
30 
+}
31 
+
32 
+
33 
+## HTTPS host
34 
+server {
35 
+  listen 0.0.0.0:443 ssl;
36 
+  listen [::]:443 ipv6only=on ssl default_server;
37 
+  server_name YOUR_SERVER_FQDN; ## Replace this with something like huginn.example.com
38 
+  server_tokens off; ## Don't show the nginx version number, a security best practice
39 
+  root /home/git/huginn/public;
40 
+
41 
+  ## Increase this if you want to upload large attachments
42 
+  ## Or if you want to accept large git objects over http
43 
+  client_max_body_size 20m;
44 
+
45 
+  ## Strong SSL Security
46 
+  ## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
47 
+  ssl on;
48 
+  ssl_certificate /etc/nginx/ssl/huginn.crt;
49 
+  ssl_certificate_key /etc/nginx/ssl/huginn.key;
50 
+
51 
+  ssl_ciphers 'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4';
52 
+  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
53 
+  ssl_prefer_server_ciphers on;
54 
+  ssl_session_cache shared:SSL:10m;
55 
+  ssl_session_timeout 5m;
56 
+
57 
+  ## See app/controllers/application_controller.rb for headers set
58 
+
59 
+  ## [Optional] If your certficate has OCSP, enable OCSP stapling to reduce the overhead and latency of running SSL.
60 
+  ## Replace with your ssl_trusted_certificate. For more info see:
61 
+  ## - https://medium.com/devops-programming/4445f4862461
62 
+  ## - https://www.ruby-forum.com/topic/4419319
63 
+  ## - https://www.digitalocean.com/community/tutorials/how-to-configure-ocsp-stapling-on-apache-and-nginx
64 
+  # ssl_stapling on;
65 
+  # ssl_stapling_verify on;
66 
+  # ssl_trusted_certificate /etc/nginx/ssl/stapling.trusted.crt;
67 
+  # resolver 208.67.222.222 208.67.222.220 valid=300s; # Can change to your DNS resolver if desired
68 
+  # resolver_timeout 5s;
69 
+
70 
+  ## [Optional] Generate a stronger DHE parameter:
71 
+  ##   sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096
72 
+  ##
73 
+  # ssl_dhparam /etc/ssl/certs/dhparam.pem;
74 
+
75 
+  ## Individual nginx logs for this huginn vhost
76 
+  access_log  /var/log/nginx/huginn_access.log;
77 
+  error_log   /var/log/nginx/huginn_error.log;
78 
+
79 
+  location / {
80 
+    ## Serve static files from defined root folder.
81 
+    ## @huginn is a named location for the upstream fallback, see below.
82 
+    try_files $uri $uri/index.html $uri.html @huginn;
83 
+  }
84 
+
85 
+  ## If a file, which is not found in the root folder is requested,
86 
+  ## then the proxy passes the request to the upsteam (huginn unicorn).
87 
+  location @huginn {
88 
+    ## If you use HTTPS make sure you disable gzip compression
89 
+    ## to be safe against BREACH attack.
90 
+    gzip off;
91 
+
92 
+    proxy_read_timeout      300;
93 
+    proxy_connect_timeout   300;
94 
+    proxy_redirect          off;
95 
+
96 
+    proxy_set_header    Host                $http_host;
97 
+    proxy_set_header    X-Real-IP           $remote_addr;
98 
+    proxy_set_header    X-Forwarded-Ssl     on;
99 
+    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
100 
+    proxy_set_header    X-Forwarded-Proto   $scheme;
101 
+    proxy_set_header    X-Frame-Options     SAMEORIGIN;
102 
+
103 
+    proxy_pass http://huginn;
104 
+  }
105 
+
106 
+  ## Enable gzip compression as per rails guide:
107 
+  ## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
108 
+  ## WARNING: If you are using relative urls remove the block below
109 
+  ## See config/application.rb under "Relative url support" for the list of
110 
+  ## other files that need to be changed for relative url support
111 
+  location ~ ^/(assets)/ {
112 
+    root /home/huginn/huginn/public;
113 
+    gzip_static on; # to serve pre-gzipped version
114 
+    expires max;
115 
+    add_header Cache-Control public;
116 
+  }
117 
+
118 
+  error_page 502 /502.html;
119 
+}